Incorporating Expert Judgement into Bayesian Network Machine Learning

We review the challenges of Bayesian network learning, especially parameter learning, and specify the problem of learning with sparse data.We explain how it is possible to incorporate both qualitative knowledge and data with a multinomial parameter learning method to achieve more accurate predictions with sparse data. 1 Review of Bayesian Network Learning Constructing a Bayesian network (BN) from data is widely accepted as a major challenge in decision-support systems. For many critical risk analysis problems, decisions must be made where there is sparse or no direct historical data to draw upon, or where relevant data is difficult to identify. The challenge is especially acute when the risks involve novel or rare systems and events [Fenton and Neil, 2012] (e.g. think of novel project planning, predicting events like accidents, terrorist attacks, and cataclysmic weather events). There are two typical categories of problems in learning BNs: one is parameter learning given a fixed graphical structure of the BN; and the other is structure learning, where the BN structure is unknown. Ideally, with sufficient data, classical learning algorithms like BDeu+MLE, PC+MLE or hybrid+MLE [Campos, 2007] can learn BNs that fit the true model in distribution and structure. However, these learning algorithms do not work when there is sparse data. To mitigate this problem, expert judgements are needed to supplement learning. In the absence of data, experts are usually required to provide strong information like causality between nodes in structure learning, and specific numerical probability values of Node Probability Tables (NPTs) or Dirichlet priors in parameter learning. Such strong judgments can easily cause bias. Studies show that experts are often overconfident in providing qualitative knowledge rather than quantitative estimations [Druzdzel and van der Gaag, 2000]. Typical qualitative knowledge are constraints that limit the number of parents of a node (in structure learning) and equality/inequality relations among parameters in parameter learning; such constraints cut the search space significantly, and help escape local maxima. Because of the potential benefits, there is an increasing research interest in incorporating constraints into structure/parameter learning. Recent reseach developments have focused on triggering the necessary automated calculations and inferences to get more accurate BNs under these constraints. For parameter learning, some approaches formulate this problem as a general constrained maximization problem, and outline the details of the classification of parameter constraint types ([Niculescu et al., 2006] and [Liao and Ji, 2009]). Unfortunately, these approaches can be extremely inefficient for BNs with a large number of parameters. Nor can they handle exterior constraints among parameters, as discussed in [Feelders and van der Gaag, 2006] and [Tong and Ji, 2008]. The work of [Tong and Ji, 2008] has limited forms of constraints, while and the work of [Feelders and van der Gaag, 2006] can only learn the parameter for binary variables. Hence, our research is focused on the development of an extended BN graphical notation, and associated algorithms, to integrate judgements provided by domain experts in a much richer and less constrained way than the current state-of-theart of modelling and tools supports.